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Stroke  rehabilitation  is  labor-intensive  and  time-consuming.  To  assist  patients  and  therapists  alike,  we 
propose  a  wearable  system  that  measures  orientation  and  corrects  arm  posture  using  vibrotactile  actu¬ 
ators.  The  system  evaluates  user  posture  with  respect  to  a  reference  and  gives  feedback  in  the  form  of 
vibration  patterns.  Users  correct  their  arm  posture,  one  DOF  at  a  time,  by  following  a  protocol  starting 
from  the  shoulder  up  to  the  forearm.  Five  users  evaluated  the  proposed  system  by  replicating  ten  dif¬ 
ferent  postures.  Experimental  results  demonstrated  system  robustness  and  showed  that  some  postures 
were  easier  to  mimic  depending  on  their  naturalness. 

©  201 1  IPEM.  Published  by  Elsevier  Ltd.  All  rights  reserved. 


1.  Introduction 

Demand  on  the  healthcare  system  rises  in  proportion  to  the 
aging  population.  Technology  can  fill  in  the  supply  gap  by  providing 
automated  systems  to  augment  healthcare  services  [1].  For  reha¬ 
bilitation,  the  advent  of  small  inertia-based  sensors  and  feedback 
devices  like  vibrotactile  actuators  could  help  improve  healthcare 
delivery. 

Micromachined  accelerometers,  gyroscopes,  and  magnetome¬ 
ters  are  sufficiently  small  to  be  attached  to  the  human  body  without 
interfering  with  movement.  Accelerometers  are  used  as  inclinome¬ 
ter  for  motion  where  acceleration  with  respect  to  gravity  are 
negligible  while  gyroscopes  measure  angular  velocity  and  pro¬ 
vide  estimate  in  orientation  change.  Magnetometers  determine  the 
local  earth  magnetic  held  vector  and  provide  additional  informa¬ 
tion  about  orientation.  These  sensors  are  packaged  together  to  form 
an  inertial  measurement  unit  (IMU),  which  is  relatively  cheap  and 
consumes  little  energy. 

IMU  had  been  employed  in  posture  measurement  and  reha¬ 
bilitation:  quantifying  hemiparesis  by  measuring  hand  path  in 
pointing  tasks  [2],  treating  idiopathic  scoliosis  [3],  and  detect¬ 
ing  and  assessing  severity  of  Parkinson’s  disease  [4].  Moreover, 
accelerometers  had  been  used  to  detect  trunk  posture  [5],  and  iner¬ 
tia  sensors  to  measure  full  body  motion  [6]  and  determine  gait 
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kinematics  [7].  Other  technologies  that  have  been  proven  effec¬ 
tive  in  stroke  patient  rehabilitation  include  functional  electrical 
stimulation  (FES)  for  artificial  activation  of  the  skeletal  muscles 
[8,9]. 

Interestingly,  sensing  user  movement  may  not  be  sufficient. 
Feedback  from  therapists  also  plays  an  important  role  in  inform¬ 
ing  patients  of  their  progress  leading  to  better  chances  of  recovery 
[10].  Feedback  can  be  audio,  visual,  tactile,  or  any  combination 
thereof.  Audio  feedback  was  used  in  [11],  where  patients  adjusted 
their  trunk  posture  depending  on  feedback  quality.  In  [12],  audio 
and  visual  cues  from  the  teacher  were  aided  by  a  robotic  suit  that 
employed  tactile  feedback  to  guide  the  movement  of  the  upper 
limb.  Vibrotactile  was  used  in  [13]  to  improve  dynamic  gait  of  the 
elderly.  Improvements  in  motion  performance  of  Tai-Chi  practi¬ 
tioners  were  realized  using  vibrotactile  feedback  complemented 
with  audio  feedback  [14].  In  [15,16],  visual  and  tactile  feedback 
were  used  to  guide  the  subjects  in  replicating  the  target  arm  pos¬ 
ture.  A  comparison  of  selected  works  is  presented  in  Table  1. 

The  works  cited  demonstrate  the  robustness  of  inertia-based 
sensors  in  measuring  limb,  trunk,  or  full  body  posture  and  the 
importance  of  feedback  in  correctly  replicating  the  desired  pos¬ 
tures.  This  paper  presents  a  system  that  uses  inertia-based  sensors 
to  measure  arm  posture  and  vibrotactile  actuators  to  guide  arm 
posture  correction.  Also  presented  is  a  protocol  in  measuring  com¬ 
plete  arm  posture  that  requires  only  two  IMUs  and  two  vibrotactile 
actuators. 

The  optimized  design  combination  of  inertia  sensors  and  vibro¬ 
tactile  feedback  to  correct  arm  posture  is  the  main  contribution 
of  this  paper.  The  system  finds  the  balance  between  number  of 
inertia  sensors  needed  and  the  number  of  joints  (DOFs)  that  has 
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Table  1 

Comparison  of  works  that  used  vibrotactile  feedback. 


Work 

Purpose 

Body  part  measured 

Measurement  device 

Vibrotactile  device 

[5] 

Balance  training 

Trunk  (mediolateral  sway) 

Microstrain  Inertia-Link  IMU 

Tactaid  VBW32  (4  units) 

[12] 

Motor  learning 

Arm  (5DOF) 

Vicon 

Tactaid  (8  units) 

[13] 

Risk  fall  indicator 

Trunk  (mediolateral  sway) 

IMU 

Tactaid  VBW32  (48  units) 

[14] 

Gesture  correction 

Upper  body  (14DOF) 

Vicon 

SHAKE 

[16] 

Motion  replication 

Arm  (5DOF) 

K-Health  IMU 

Solarbotics  VPM2  (3  units) 

to  be  measured.  The  key  is  the  protocol  that  allows  optimum 
conveyance  of  feedback  information  to  the  users  while  using  the 
minimum  number  of  vibrotactile  motors.  While  other  works  have 
used  inertia-based  sensors  and  vibrotactile  feedback  in  various 
applications,  ours  focuses  particularly  on  rehabilitation,  where 
accurate  posture  measurement  and  timely  feedback  could  aid  the 
patient  in  regaining  lost  muscular  abilities. 

The  rest  of  the  paper  is  organized  as  follows.  Section  2  intro¬ 
duces  the  mathematical  foundations  of  using  IMU  to  measure  and 
correct  arm  posture.  Section  3  discusses  the  method  for  guiding 
arm  posture  correction  while  Section  4  presents  the  system  design 
and  method  implementation.  Section  5  details  the  experimental 
setup  and  Section  6  presents  the  experimental  results.  Section  7 
concludes  the  paper  and  lists  future  work. 


2.  Measuring  and  correcting  arm  posture 

2.1.  Working  principle  of  the  IMU  sensor 

Each  inertial  measurement  unit  (IMU)  sensor  has  an  accelerom¬ 
eter,  a  magnetic  sensor  and  two  gyroscopes  inside.  The  IMU’s  roll 
(<p)  is  the  angle  measured  around  the  x-axis  (parallel  to  the  ground), 
its  pitch  (p)  is  the  angle  around  they-axis  (parallel  to  ground),  and 
its  yaw  ( 0 )  is  the  angle  around  the  z-axis  (parallel  to  gravity).  The 
IMU  with  its  orientation  axes  is  shown  in  Fig.  1(a). 

The  orientation  of  the  IMU  [(pacc(t)  Pacc(f)  0acc(Ol»  is  derived  from 
the  angular  velocities  [cox  ooz\  about  the  three  axes  of  the  gyro¬ 
scope.  The  angular  displacements  are  calculated  by  integrating  the 
angular  velocities: 


(a)  The  inertial  measurement  unit  (b)  The  vibrotactile 


(IMU)  sensor,  with  its  axes  shown,  actuators. 


Given  the  angular  value  at  (t  -  1 )  and  At,  the  numerical  approx¬ 
imation  becomes: 

< Pgyroit )  =  (Pgyro{t  —  1)  +  COxAt 

Pgyro(t)  =  Pgyro(t  —  1)  +  COyAt  (2) 

@gyro(t )  =  OgyW(t  —  1)  +  COzAt 

We  use  the  accelerometer  measurement  to  correct  for  the  gyro¬ 
scope  integration  error.  Finally  we  obtain  the  rotation  matrix: 


*Ti 

*T2 

**13 

*21 

**22 

**23 

*31 

**32 

**33 

where, 

ru  =cos (<p)  cos (p) 

r12  =  -  sin(<p)  cos(0)  +  cos(^)  sin(p)  sin(0) 
ri3  =  sin(<p)  sin(0)  +  cos{cp)  sin(p)  cos(0) 
r2\  =  sin(p)  cos(p) 

r22  =  cos(<p)  cos(0)  +  sin(<p)  sin(p)  sin(0)  (4) 

r23  =  -  cos(^)  sin(0)  +  sin(<p)  sin(p)  cos(0) 

r31  =  -  sin(p) 

r32  =  cos(p)  sin(0) 

r3 2  =  cos(p)  cos(0). 

2.2.  Arm  model 

Fig.  2  illustrates  the  two  arm  segments  (upper  arm  and  forearm); 
each  has  distinct  curvature  and  range  of  motion.  Arm  motion,  par¬ 
ticularly  of  the  elbow  and  the  wrist,  can  be  modeled  as  a  compound 
flexible  pole  (CFP).  A  flexible  pole  is  capable  of  bending  and  rotat¬ 
ing  in  three  dimensions  with  no  significant  deformation  along  its 
length. 

Each  arm  segment  is  modeled  as  a  CFP,  a  rigid  body  with 
an  IMU  sensor  attached,  and  treated  as  massless  for  orientation 


Fig.  2.  A  compound  flexible  pole  model  divides  the  arm  into  two  main  sections: 
the  forearm  and  the  upper-arm.  The  inset  shows  where  the  two  IMU  sensors  are 
mounted  on  the  arm. 


Fig.  1.  The  sensors  and  actuators. 
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measurements.  Modeling  the  arm  as  such  allowed  the  expres¬ 
sion  of  the  arm  motion  in  the  more  intuitive  form  of  rotation, 
flexion-extension  and  lateral  bending.  Moreover,  this  arrangement 
enables  measurement  of  the  whole  arm  at  two  strategic  points:  the 
shoulder  and  the  wrist. 


2.3.  Orientation  and  position  measurement 


The  orientation  of  the  upper  arm  (<pu(t)  pu(t)  0u(t))  is  derived 
from  the  elbow  measurements.  Using  Eq.  (2),  elbow  (xu,yu,  zu)  and 
wrist  (*/,y/,  Zf)  positions  become: 
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where  Lf  is  the  forearm  length  and  Lu  is  the  upper  arm  length. 

Moreover,  given  that  the  upper  arm  and  the  forearm  form  a 
triangle  with  vertices  at  the  wrist,  elbow,  and  shoulder: 


xj  +yj  +zj  =Ll+Lj  -  2LuLf  cos(ye) 

The  joint  angle  of  elbow  ye  is  derived  as: 

Ye(t)  =  arccos(-cos(0/(t)-0u(t))cos(p/(t))cos(pu(t)) 
-sin(p/(0)sin(pu(t))) 


(7) 


(8) 


Thus,  the  elbow  angle  ye  is  directly  related  to  the  upper  arm 
[pu(t)  0„(t)]  and  the  forearm  [pj{t)  Oj(t)]  orientations. 

2.4.  Coordinate  calibration  with  respect  to  the  elbow  joint  angle 

Each  IMU  sensor  is  referenced  with  respect  to  the  shoulder  such 
that  the  coordinate  system  of  each  arm  segment  and  the  attached 
IMU  are  coupled  together;  the  x-axis  lies  along  the  arm  length  and 
the  z-axis  is  against  gravity  vector.  To  correct  for  the  IMU  sen¬ 
sor’s  displacement  offset,  users  straighten  their  whole  arm,  making 
sure  that  their  elbow  reaches  180°.  Then,  the  orientation  of  the 
inertia  sensors  are  recorded  and  the  rotation  matrices  of  the  dis¬ 
placement  offsets  are  calculated  as  R „ 1  (0)  and  Rj1  (0).  The  rectified 

rotation  matrices  of  the  elbow  and  the  wrist  become  Ru(t)R^{ 0) 
and  Rf{t)Rj^{ 0),  respectively. 

3.  Posture  correction  protocol 

A  user’s  current  posture  and  a  reference  posture  are  consid¬ 
ered  matched  if  and  only  if  the  orientation  difference  between 
them  is  within  permissible  range.  Generally,  there  are  two  ways  to 
control  posture  orientation:  orientation  trajectory  control  and  posi¬ 
tion  control.  Here  we  propose  to  use  orientation  posture  matching 
approach,  wherein  users  adjust  their  posture  in  the  following  order: 

1.  3D  orientation  (yaw,  pitch,  roll)  of  the  upper  arm; 

2.  Elbow  joint  angle;  and 

3.  Forearm  orientation. 


The  orientation  deviation  A 6U  of  the  upper  arm  yaw  axis  is 
derived  from  the  orientation  of  the  reference  posture  -  upper  arm 


Wu,r  Pu,r  0u,r]  and  forearm  \cpf  r  pfr  0f  r].  Then  the  user  moves  his  arm 
to  reach  the  posture  with  A 0U  within  the  permissible  range  of  the 
reference  posture.  The  same  method  is  applied  to  adjust  A pu  and 
A  cpu- 

After  correcting  the  orientation  of  the  upper  arm,  the  elbow 
angle  deviation  A  ye  is  corrected,  followed  by  the  forearm  roll  devi¬ 
ation  A(pf.  To  achieve  this,  the  user  moves  his  arm  until  his  elbow 
reaches  the  posture  with  A ye  within  the  permissible  range.  Lastly, 
the  user  repeats  the  same  process  for  the  forearm  roll  to  attain  the 
posture  with  A<pf  within  the  acceptable  range. 

To  summarize,  users  need  to  adjust  the  orientation  of  0U,  pu,<Pu, 
ye  and  cpp  in  this  particular  sequence,  when  correcting  for  the  refer¬ 
ence  posture.  With  this  protocol,  we  were  able  to  use  the  minimum 
number  of  vibrotactile  actuators  for  feedback. 

4.  System  design  and  implementation 

The  main  components  of  the  system  are  the  inertia  sensors 
(IMU)  and  the  vibrotactile  actuators. 

4.1.  Integrated  vibrotactile  inertial  measurement  unit 

A  wearable  vibrotactile  inertial  measurement  unit  (VIMU)  sys¬ 
tem  integrates  three  major  components:  a  digital  signal  controller 
(DSC),  two  inertia-based  sensors  (IMUs),  and  a  vibrotactile-based 
motion  indicator. 

4.1.1.  Digital  signal  controller 

The  DSC,  which  controls  the  vibrotactile  actuators,  is  a  low- 
power  8-bit  ATMEL  ATmegal  68  Micro  Controller  Unit  (MCU)  based 
on  the  AVR  enhanced  RISC  architecture.  In  our  prototype  the  MCU  is 
embedded  in  an  Arduino  Duemilanove,  an  I/O  development  board. 
It  is  both  computing  platform  and  development  environment  for 
the  system’s  software. 

4.1.2.  Inertial  measurement  unit  (IMU) 

Each  IMU  provides  drift-free  orientations  and  calibrated  3  DOF 
linear  accelerations  (from  the  accelerometer),  3  DOF  angular  veloc¬ 
ities  (from  the  gyroscopes)  and  3  DOF  magnetic  data  (from  the 
magnetometer).  The  accelerometer  is  a  dual-axis  low  g  MEMS- 
based  capacitive  ADXL320  from  Analog  Devices.  It  is  capable  of 
measuring  linear  acceleration  signals  over  a  bandwidth  of  60  Hz 
and  has  an  effective  sensing  range  of  ±2  g/±6  g.  The  magnetometer 
is  a  tri-axis  magnetic  field  sensing  module  HMC1053  from  Honey 
Well.  It  has  a  measurement  range  of  ±6  Gauss  and  capable  of  mea¬ 
suring  the  geomagnetic  vector.  The  pair  of  gyroscopes,  dual-axis 
gyroscopes  from  Silicon  Sensing  System  (Japan),  measures  tri-axis 
angular  velocities  with  a  measurement  range  of  ±300  °/s  in  yaw, 
and  ±500  °/s  in  roll  and  pitch. 

We  developed  the  software  to  capture  and  process  the  IMU  data 
into  rotation,  flexion-extension  and  lateral  bending  parameters. 
The  data  is  transferred  to  a  computer  via  wireless  channels,  making 
the  whole  system  portable. 

4.1.3.  Vibrotactile  actuator  (tactor) 

The  feedback  unit  uses  vibrotactile  actuators  (tactors),  small 
transducers  designed  to  optimize  skin  response  to  vibration.  The 
tactors  are  small  flat  type  (VPM2)  and  consume  little  power  but 
provide  adequate  vibrating  strength.  They  are  manufactured  by 
Solarbotics  (Calgary,  Canada)  and  shown  in  Fig.  1(b).  Each  tactor 
is  activated  at  the  80-250  Hz  frequency  range  and  converts  elec¬ 
trical  energy  into  mechanical  displacement  by  using  DC  power  to 
rotate  an  eccentric  weight.  With  a  3  V  driving  voltage  each  tactor 
is  able  to  generate  1  g  (9.8  m/s2)  acceleration  vibration  level,  which 
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(a)  Illustration  of  the  finite  state  machine  (b)  The  vibration  pattern  associated  with  each 
that  determines  the  transition  for  correct-  controller  state, 
ing  posture  orientation  parameters. 

Fig.  3.  Details  of  the  feedback  system  design. 


is  higher  than  the  discrimination  thresholds  of  the  forearm  and  the 
upper  arm,  0.6  g  and  0.8  g,  respectively  [17]. 

4.2.  Placement  oftactors  on  the  human  body 

Based  on  the  study  of  skin  reaction  to  vibrotactile  feedback,  two 
adjacent  vibrotactile  actuators  must  have  a  minimum  critical  dis¬ 
tance  in  order  for  their  vibration  to  be  distinguishable  [18-24]. 
This  threshold  varies  depending  on  tactor  placement.  For  forearm 
placement,  the  minimum  threshold  for  both  men  and  women  is 
about  40  mm  while  for  the  forearm,  the  threshold  is  48  mm  for  men 
and  42  mm  for  women.  For  optimal  perception  in  the  forearm,  the 
threshold  can  be  increased  to  50  mm  [18]. 

4.3.  Two-tactor  feedback  to  guide  motion  of  a  5  DOF  arm 

In  this  section  we  define  the  arm  posture  correction  protocol 
when  used  with  a  2-tactor  feedback  setup.  As  of  this  point,  it  is 
assumed  that  arm  posture  is  obtainable  through  the  IMUs  and  the 
thresholds  for  each  posture  parameter  have  already  been  defined. 

Fig.  3(a)  shows  the  complete  posture  correction  process  where 
each  posture  parameter  0U,  Pu,<Pu,  Ye .  and  (pf  follow  the  same  con¬ 
trol  scheme.  The  controller  goes  through  the  states  Si,  S2,  and  S3 
while  minimizing  the  orientation  deviation  between  user  and  ref¬ 
erence  postures.  The  states  are  defined  as  follow: 

51  “Correct  target  direction”  indication:  At  this  state,  the  tactor 
vibrates  at  a  constant  magnitude  as  the  user  moves  his  arm  in 
search  for  the  correct  direction.  When  the  limb  starts  moving  to  the 
correct  direction,  the  controller  will  release  a  diminishing  linear 
proportional  vibration  pattern. 

52  “Continue  moving  to  target”  indication:  The  controller  will 
examine  in  real-time  if  the  orientation  deviation  falls  within  a 
small  range,  i.e.,  the  user  is  not  radically  changing  his  direction. 


If  deviation  is  within  range,  the  controller  provides  a  constant 
vibration  pattern  to  let  the  user  know  that  he  must  continue 
moving  in  the  same  direction  at  constant  speed. 

S3  “Closing  in  on  the  target”  indication:  Once  the  orientation 
deviation  is  within  the  threshold  value,  the  controller  will  pro¬ 
duce  a  burst-like  pattern  to  indicate  that  the  subject  is  close 
to  the  reference.  Once  the  parameter  orientation  is  within  the 
permissible  range,  the  tactor  stops  vibrating  and  the  controller 
waits  to  process  the  next  parameter. 

The  vibration  pattern  associated  with  each  state  is  illustrated  in 
Fig.  3(b). 

By  correcting  the  posture  in  series,  our  system  can  afford  to  use 
only  two  tactors  for  feedback.  The  feedback  toggles  between  the 
two  tactors  to  provide  the  necessary  information  to  the  subjects. 
That  is,  given  two  tactors  Ta  and  Tb:  during  the  correction  of  the 
first  parameter  0U,  Ta  will  provide  the  feedback.  Then,  for  the  next 
parameter  pu ,  Tb  will  provide  the  feedback.  For  the  third  parameter, 
(pu,  feedback  reverts  to  Ta.  For  the  last  two  parameters,  ye  and  cpp 
feedback  is  handled  by  Tb  and  Tfl,  respectively. 

5.  Experimental  setup 

5.1.  Subjects 

Five  students  from  the  university,  with  ages  ranging  from  20 
to  27  years,  volunteered  to  participate  in  the  experiments.  All  were 
healthy  and  without  any  medical  condition  that  could  have  affected 
their  tactile  sensitivity. 

5.2.  Hardware  setup 

The  two  tactors  were  placed  around  the  subject’s  arm.  The 
distance  between  the  two  tactors  was  larger  than  the  two-point 


(a)  The  vibrotactile  unit. 


(b)  Sensors  and  vibrotactile  unit 


Fig.  4.  The  location  of  the  vibrotactile  actuators  is  encircled  in  (a).  The  relative  position  of  the  sensors  and  the  feedback  unit  on  the  arm  (b). 
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Fig.  5.  The  ten  postures  used  in  the  experiments. 


discrimination  threshold  for  both  male  and  female  [18].  The 
factors,  secured  in  a  holder,  were  attached  to  elastic  bands  to  fit 
the  arm  of  different  subjects.  The  materials  chosen  for  the  factor 
holders  were  thin  and  soft  to  ensure  vibration  detection.  The  whole 
vibrotactile  unit  was  light  weight,  comfortable  to  use,  and  easy  to 
wear.  Fig.  4(a)  shows  the  inner  side  of  the  vibrotactile  feedback 
unit.  The  digital  controller  is  mounted  on  the  other  side  of  the 
strap. 


5.3.  General  procedure 

The  experiment  required  a  master,  who  establishes  the  refer¬ 
ence  posture,  and  students,  who  must  copy  the  reference  posture. 
Students  were  not  allowed  to  see  the  master’s  posture;  they  had 
to  rely  solely  on  the  vibrotactile  feedback  to  correct  their  posture. 
Prior  to  each  experiment,  students  practiced  several  times  until 
they  became  familiar  with  interpreting  the  vibrotactile  signal  and 
moving  to  the  indicated  direction.  The  training  process  required 
the  students  to  turn  their  upper  arm  left  and  right  around  the  yaw 
axis,  up  and  down  around  the  pitch  axis,  and  to  rotate  around  the 
roll  axis. 

For  each  experiment,  the  following  procedures  were  followed: 


1.  The  master  wears  two  IMUs  and  assumes  the  target  posture. 

2.  A  student  wears  two  IMUs  and  one  vibrotactile  unit  as  shown  in 
Fig.  4(b). 

3.  The  student  faces  in  the  same  direction  as  the  master  and  then 
stretches  his  whole  arm  to  assume  the  starting  position. 

4.  The  student  holds  his  upper  arm  for  about  half  a  second  to  check 
whether  factor  A  is  giving  a  bursting  vibration  pattern  before 
correcting  for  the  upper  arm’s  yaw  orientation.  If  the  factor  is 
already  vibrating  in  a  burst  pattern,  then  the  current  upper  arm 
position  is  already  within  the  correct  yaw  angle  and  only  needs 
minor  adjustment.  Otherwise,  the  student  must  start  correcting 
for  the  pitch  orientation. 

5.  The  student  must  continue  moving  his  arm  in  the  current  direc¬ 
tion  of  motion  if  a  decreasing  vibration  strength  is  sensed; 
otherwise  the  student  must  move  his  arm  to  the  opposite  direc¬ 
tion. 

6.  The  student  must  slow  down  to  get  to  the  target  position  when 
a  burst  vibration  pattern  is  sensed.  When  target  is  reached,  the 
vibration  signal  will  toggle  from  factor  A  to  factor  B,  indicating 
the  beginning  of  correction  of  the  next  parameter. 


7.  Steps  4-6  are  repeated  for  the  next  posture  parameter  until  all 
parameters  are  exhausted. 

To  be  able  to  compare  posture  among  different  subjects,  the 
master’s  postures  were  recorded.  Thus,  for  all  the  experiments  the 
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Fig.  6.  Results  of  replicating  Posture  1  by  five  students,  (a)  Upper  arm  yaw;  (b)  upper 
arm  pitch;  (c)  upper  arm  roll;  (d)  elbow  joint;  (e)  forearm  roll. 
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Fig.  7.  Average  correction  time  of  the  ten  postures  according  to  individual  DOF.  (a)  Upper  arm  yaw;  (b)  upper  arm  pitch;  (c)  upper  arm  roll;  (d)  elbow  angle  and  forearm  roll. 
SI,  S2,  and  S3  refer  to  the  states  as  illustrated  in  Fig.  3  (b). 


Fig.  8.  Trajectory  of  Postures  1  and  9.  (a)  Trajectory  of  the  upper  arm’s  yaw;  (b)  trajectory  of  the  upper  arm’s  pitch;  (c)  trajectory  of  the  upper  arm’s  roll;  (d)  trajectory  of  the 
elbow  angle;  (e)  trajectory  of  the  forearm’s  roll. 
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Table  2 

Average  user  correction  time  for  the  ten  postures. 


Posture 

Correction  time  (s) 

1 

36.67  ±  6.53 

2 

28.97  ±  3.62 

3 

27.54  ±  4.66 

4 

27.29  ±  5.39 

5 

41.25  ±  5.02 

6 

58.61  ±  2.56 

7 

53.30  ±  2.86 

8 

72.06  ±  9.40 

9 

73.32  ±  7.70 

10 

38.97  ±  7.35 

pre-recorded  master’s  postures  were  just  loaded  from  computer 
memory. 

6.  Results  and  discussion 

To  check  the  feasibility  of  our  system  ten  postures  were  tested 
based  on  arm  movement  in  3D  space,  which  had  been  divided  into 
eight  zones.  The  postures  are  shown  in  Fig.  5. 

The  first  four  zones  are  at  the  anterior:  left-up,  right-up,  left- 
down  and  right-down  (Postures  1,  2,  3,  and  4).  Similarly,  there  are 
four  zones  at  the  posterior  (Postures  5,  6,  7,  and  8).  Two  additional 
postures  outside  of  the  defined  zones  were  also  tested.  In  Posture 
9,  part  of  the  forearm  passes  through  the  left  side  of  the  body.  This 
posture  seldom  happens  in  daily  activity  except  when  stretching 
the  arm  during  exercise  warm-up.  Although  Posture  10  is  a  com¬ 
mon  one,  the  forearm  is  placed  higher  than  the  head,  which  requires 
additional  effort  to  reach.  Thus,  each  of  the  ten  postures  stands  for 
some  unique  spatial  orientation  of  the  arm.  In  particular,  orienta¬ 
tions  of  Postures  6-9  are  special  because  they  need  more  effort  to 
achieve,  usually  through  extra  muscle  exertion. 

Fig.  6  shows  the  result  of  replicating  Posture  1.  The  upper  arm’s 
yaw,  pitch  and  roll  angles  for  this  posture  were  148°,  26°,  and  53°, 
respectively.  The  threshold  for  successful  mapping  was  set  at  ±5°. 
The  graphs  show  that  all  five  subjects  were  able  to  move  towards 
the  target  posture  within  30  s.  This  result  indicates  that  the  subjects 
were  able  to  sense  in  a  timely  manner  the  vibration  pattern,  which 
told  them  when  to  continue  moving  in  a  particular  direction  and 
when  to  stop. 

The  motion  speed,  represented  by  the  graph’s  slopes,  varies  from 
subject  to  subject.  Looking  at  Subject  2  (data  labeled  S2)  in  Fig.  6(a), 
the  graph  shows  that  he  was  moving  faster  compared  to  the  other 
subjects.  Flowever,  due  to  his  speed  he  overshot  the  target  angle 
and  went  past  the  “closing  in  on”  state  (state  S3).  Luckily,  he  felt  the 
quick-burst  vibration  pattern  which  quickly  made  him  adjust  his 
movement  and  went  back  to  state  S3.  After  some  fine  adjustments, 
he  reached  the  desired  yaw  angle  of  148  ±5°.  His  whole  posture 
correction  process  took  15  s. 

Fig.  6(d)  and  (e)  also  shows  the  posture  replication  for  the  elbow 
and  the  forearm.  The  elbow  and  forearm  roll  reference  angles  were 
171°  and  120°,  respectively,  and  the  threshold  was  also  set  to  ±5°. 
For  Subjects  1-5,  the  initial  elbow  angles  were  170°,  166°,  154°, 
170°,  and  171°.  With  the  exception  of  Subject  3,  all  the  other  sub¬ 
ject’s  elbow  angle  were  within  the  threshold  (the  lines  for  the  other 
subjects  are  too  short  that  they  are  nearly  invisible  in  the  graphs). 
Nevertheless,  it  took  less  than  5  s  for  Subject  3  to  correct  his  elbow 
angle.  The  results  for  mapping  the  forearm’s  roll  angle  were  similar, 
all  subjects  were  able  to  reach  the  target  posture  within  5  s. 

Table  2  lists  the  average  time  to  reach  the  desired  posture  for 
each  subject.  For  postures  that  appear  natural,  such  as  Postures  1, 
2, 3, 4,  and  10,  the  subjects  reached  them  within  40  s.  However,  the 
subjects  had  to  spend  more  time  on  Postures  6,  7,  8,  and  9,  with 
mapping  time  ranging  from  50  to  70  s. 


Fig.  7  shows  the  average  correction  time  of  all  users  for 
each  arm  posture  parameter.  The  correction  time  for  the  upper 
arm  (Fig.  7(a-c))  was  further  divided  into  different  phases  that 
correspond  to  the  vibration  feedback  control  states.  Upper  arm 
yaw  mapping  required  more  time  during  the  S2  phase,  while  the 
upper  arm  pitch  mapping  required  more  time  during  the  S3  phase. 
Compared  to  correction  of  the  elbow  joint  angle,  correction  of  the 
forearm’s  roll  took  longer  (Fig.  7(d)). 

In  Fig.  8,  the  trajectory  of  Posture  1  is  smoother  than  that  of 
Posture  9.  The  subject  took  more  time  in  correcting  the  forearm’s 
roll  angle  as  shown  in  Posture  9.  However,  it  is  reasonable  to  assume 
that  in  order  to  complete  Posture  9,  the  subject  had  to  stretch  his 
arm  longer  and  farther. 

7.  Conclusion 

In  this  paper,  we  presented  a  guidance  system  that  helps  sub¬ 
jects  replicate  a  reference  arm  posture  using  inertia-based  sensors 
and  vibrotactile  feedback.  Our  system  supports  real-time  arm  pos¬ 
ture  measurement  and  orientation  feedback  generation.  We  tested 
our  system  with  several  subjects  using  ten  postures  that  represent 
different  arm  configurations  in  3D  space.  All  the  subjects  man¬ 
aged  to  reach  the  target  posture,  with  varying  performance,  usually 
depending  on  the  difficulty  of  the  posture. 

Our  future  work  includes  several  tasks.  First,  the  protocol  for 
posture  correction  needs  modification  if  dynamic  body  movements 
will  be  used.  As  some  postures  can  be  difficult  to  replicate,  addi¬ 
tional  feedback  mode  might  be  necessary  to  maintain  fast  posture 
correction.  Testing  of  our  system  in  actual  rehabilitation  setting 
with  stroke  patients  is  currently  under  consideration. 

The  posture  correction  system  that  we  have  developed  is 
small,  wearable,  and  non-intrusive,  making  it  ideal  for  therapy 
sessions  and  for  extended  wear.  Patients  can  use  the  system  to 
undergo  rehabilitation  with  minimal  therapist  supervision  (such 
as  in  remote  rehabilitation),  freeing  therapist’s  time  while  empow¬ 
ering  the  patients  to  take  control  of  their  own  therapy. 
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